The present invention relates generally to a compressed picture data editing apparatus and method and recording medium for storing instructions to perform such editing and to store such edited data, and more particularly to a compressed picture data editing apparatus and method in which two compressed picture data streams are spliced to each other at an arbitrary splice point to generate one continuous compressed picture data stream, and a recording medium on which the continuous compressed picture data stream is stored.
Recently, digital transmission systems for moving pictures such as a digital broadcasting system, a visual telephone system, etc. have been developed. In such digital transmission systems in general, moving picture data is compressed according to the MPEG-2/Video method. MPEG is an abbreviation for the xe2x80x9cMoving Picture coding Experts Groupxe2x80x9d of the ISO/IEC JTC1/SC29 (International Organization for Standardization/International Electrotechnical Commission, Joint Technical Committee 1/Sub Committee 29). An ISO 11172 standard is provided for an MPEG Phase 1, and an ISO 13818 standard is provided for an MPEG Phase 2. In these international standards, ISO 11172-1 and 13818-1 are provided for system multiplexing, ISO 11172-2 and 13818-2 are provided for video coding, and ISO 11172-3 and 13818-3 are provided for audio coding.
According to the MPEG-2/Video standard (ISO 13818-2), when decoding a compressed picture data input, in order to prevent an input buffer located at the front part of a decode circuit from underflowing or overflowing, the capacity of the input buffer should be considered, and the amount of compression during encoding should be controlled so that upon decoding the input buffer can accommodate the supplied data amount without underflowing or overflowing. When encoding according to the MPEG-2/Video standard, a virtual buffer for monitoring and controlling the amount of code that would be generated during decoding is defined as a VBV (Video Buffering Verifier) buffer, and the capacity of this VBV buffer is defined as VBV buffer size (vbv_buffer_size). The function of the VBV buffer will be further described with reference to FIG. 14, which depicts the amount of data stored in an input buffer provided upstream of a decode circuit when the buffer is supplied with input MPEG streams. The horizontal axis of FIG. 14 represents time along which decoding times (t101, t102, t103, . . . ) for various pictures are shown while the vertical axis represents the amount of data (bit occupancy) stored in the input buffer. The input buffer will store the MPEG streams compressed according to the MPEG-2/Video Method sequentially at their respective bit rates.
During a decoding procedure, the data corresponding to a first picture is extracted from the VBV buffer by the decoder at a time t101. This time t101 includes a delay (vbv_delay) from a time t100, the time at which the supply of the picture data is started, to allow for the data amount stored in the input buffer to reach an initial predetermined amount so that all of the data associated with the first few pictures can be loaded into the VBV buffer. Data representing the length of the delay (vbv_delay) is included as part of the MPEG stream in a picture layer. The time is represented by a clock of 90 kHz, for example.
During decoding, the amount of data corresponding to a picture to be extracted from the VBV buffer by the decode circuit to be decoded is the sum of the amount of data comprising the picture (picture_size), the amount of data comprising a picture start code (picture_start_code), and the amount of data comprising a sequence header (sequence_header). The amount of data resulting from the addition of picture_size, picture_start_code and sequence_header will be referred to as xe2x80x9cimage size (image_size)xe2x80x9d.
After time t101 and the extraction of the data of the first image to be decoded from the VBV buffer, the input buffer will be sequentially supplied with a succession of MPEG data streams at a predetermined bit rate corresponding to subsequent transmitted pictures. Also decoding of sequential pictures takes place at times t102, t103, . . . , tn, tn+1, . . . , xcex94DTS being a time interval between decode procedures. At each time t102, t103, . . . , tn, tn+1, . . . , an amount of data corresponding to the image size of each picture is extracted by the decode circuit from the VBV buffer for decoding. Thus, the input buffer provided upstream of the decode circuit will store the MPEG streams received sequentially at their respective bit rates while an amount of data corresponding to image size of each picture is extracted by the decode circuit at each of the decoding times (t101, t102, t103, . . . ).
If the total amount of data of the supplied MPEG streams exceeds the total amount of data corresponding to the image sizes of the pictures extracted from the VBV buffer at the decoding times, that is more data is added to the VBV buffer than is removed from it, the input buffer will eventually overflow when this excess data exceeds the size of the VBV buffer. If the total amount of data of the image sizes of the pictures extracted for decoding exceeds the total amount of data of the supplied MPEG streams, that is less data is added to the VBV buffer than is removed from it, the input buffer will underflow because more data will be taken out of the VBV buffer than will be input to it. To avoid overflowing or underflowing the VBV buffer, the MPEG-2/Video (ISO 13818-2) standard prescribes that when encoding MPEG streams, the capacity of an input buffer provided upstream of a decode circuit during decoding should be modeled as a VBV buffer size (vbv_buffer_size) to allow for control of the amount of encoded data produced during encoding. The model input buffer is thus designed to accommodate an appropriate amount of data during encoding so that during decoding, the amount of data contained in the VBV buffer will remain within a predetermined range.
Generally, at a broadcasting station or the like, in order to transmit a moving picture sequence that has been compressed according to the MPEG-2 system, often two or more sets of moving picture data are edited and spliced together to form one set of moving picture data. For example, at the broadcasting station, moving picture data of a movie may be edited to insert a commercial into the movie as shown in FIG. 15. Each of the commercials includes an amount of data that allows the commercial to run for a short period of time. When conventionally performing such an edit, for real-time transmission and for convenience in editing, base-band moving picture data are not handled during editing. Rather, compressed MPEG streams are processed during editing without decoding.
First to third conventional alternatives for performing such editing will now be described. In each of the conventional alternatives, a first picture data stream compressed according to the MPEG-2/Video Method (xe2x80x9cmain streamxe2x80x9d hereinafter) has spliced thereto a second picture data stream also compressed according to the MPEG-2/Video Method (xe2x80x9csub-streamxe2x80x9d hereinafter) to generate one spliced data stream. The conventional first to third alternatives will be described with reference to FIGS. 16 to 18, which show the amount of data in the VBV buffer attributable to the main stream, the amount of data in the VBV buffer attributable to the sub-stream, and the amount of data in the VBV buffer attributable to the spliced stream, respectively.
First, referring to FIG. 16, the conventional first alternative will be described. As is shown in FIG. 16A, a splice-out point at which the main stream is to be spliced out is defined at as a time (t111). At this splice-out point, the supply of data to the VBV buffer from the main stream will end. In the first conventional alternative, a splice-in point of the sub-stream is defined at a time (t121). At this splice-in point the supply of the sub-stream to the VBV buffer will start, as shown in FIG. 16B. According to the first conventional alternative, the splice-out and splice-in points of the main stream and sub-stream, respectively, are spliced to each other at the time t111 so that the picture in the main stream is extracted by a decode circuit until a time t112 and the picture in the sub-stream is extracted by the decode circuit after the time t112 to generate one spliced stream, as shown in FIG. 16C, for example. In the case of the spliced stream thus generated according to the first conventional alternative, however, the difference between the total data amount of the supplied picture and the total data amount of the extracted picture may exceed the VBV buffer size so that the VBV buffer will possibly overflow as indicated at the hatched portion in FIG. 16C.
Next, the second conventional alternative will be described, making reference to FIG. 17. As is shown in FIG. 17A, a splice-out point of the main stream is defined at a time (t131) at which the last picture from the mainstream will be extracted by the decode circuit. In this second conventional alternative, a splice-in point of the sub-stream is defined at a time (t141) at which time the amount of data contained in the VBV buffer will be the same as the amount of data contained in the VBV buffer at the splice-out point of the main stream, as shown in FIG. 17B. In the second conventional alternative, the splice points of the main stream and sub-stream, respectively, are spliced to each other at the time t131 so that the data associated with the picture of the main stream is extracted from the buffer by a decode circuit until a time t131, and thereafter the data associated with the picture in the sub-stream is extracted by the decode circuit to generate one spliced stream, as shown in FIG. 17C, for example. In this second conventional alternative, however, since the first picture in the sub-stream is extracted at a time at which a next picture would have been extracted from the main stream, if provided (next_vbv_delay), the total data amount of the extracted picture data from the VBV buffer may be larger than the total amount of data that has been supplied to the VBV buffer as shown in FIG. 17C. The VBV buffer may underflow. More particularly, since the VBV delay of the next picture after the last picture to be extracted from the main stream (next_vbv_delay) is shorter than the delay of the first picture to be extracted from the sub-stream (first_vbv_delay), the VBV buffer may underflow.
Finally, the third conventional alternative will now be described with reference to FIG. 18. As shown in FIG. 18A, a splice-out point of the main stream is taken as a time (t151) at which a last picture will be extracted from the VBV buffer by the decode circuit and the amount of data in the VBV buffer is zero. In this third conventional alternative, a splice-in point of the sub-stream is taken as a time (t161) at which time the supply of the sub-stream data to the VBV buffer will start, as shown in FIG. 18B. The splice points of the main stream and sub-stream are spliced to each other at the time t151 so that the picture supplied from the main stream is extracted from the VBV buffer by a decode circuit until a time t151. Thereafter, the picture supplied from the sub-stream is extracted by the decode circuit after the time t152 to generate one spliced stream, as seen from FIG. 18C, for example. In this third conventional alternative, since the supply of the sub-stream data starts after the last picture supplied by the main stream has been extracted from the VBV buffer, the VBV buffer will not overflow or underflow. However, there is a time zone (between t151 and t152) in which the decode circuit cannot extract any picture from the VBV buffer. Similarly, no data is supplied to the VBV buffer from the time (t150) at which the last picture from the main stream data is supplied until the time (t151) at which the supply of the sub-stream data starts. Therefore, the third conventional alternative does not allow for continuous provision of MPEG stream data to the decoder. The decoder is unable to properly process the MPEG stream, and as a result does not allow for stable decoding of MPEG picture data.
Therefore, it would be beneficial to provide an editing apparatus and method that overcome the drawbacks of the prior art.
Accordingly, it is an object of the invention to provide an improved editing apparatus and method that overcomes the above-mentioned drawbacks of the prior art.
Another object of the invention is to provide an improved compressed picture data editing apparatus and method adapted to generate one continuous compressed picture data stream by splicing two data streams of compressed picture data to each other at an arbitrary splice point in such a manner that an input buffer of a decoder will not overflow, underflow, or otherwise break down and in which the decoder is not constrained in its ability to process the spliced data stream.
A further object of the invention is to provide a recording medium for storing a continuous compressed picture data stream generated according to the invention.
Still other objects and advantages of the invention will in part be obvious and will in part be apparent from the specification and the drawings.
In accordance with the invention, a compressed picture data editing apparatus is provided for generating a continuous compressed picture data by splicing an arbitrary splice point of a first compressed picture data stream (first elementary stream) and an arbitrary splice point of a second compressed picture data stream (second elementary stream). The first and second elementary streams are spliced by inserting between the arbitrary splice point of the first elementary stream and the splice point of the second elementary stream a compressed picture data inserted stream including a predetermined compressed picture data and a pseudo data. A bit occupancy in a virtual buffer is determined for the first elementary stream at the arbitrary splice point, which controls an amount of code generation during encoding. A bit occupancy is also determined in the virtual buffer for the second elementary stream at the arbitrary splice point, and for an elementary inserted stream. The splicer is controlled based on the detected bit occupancies and the amount of compressed picture data in the elementary inserted stream and the amount of required pseudo data to make the bit occupancy of the elementary inserted stream in the first virtual buffer equal to the amount of compressed picture data in the first elementary stream in the virtual buffer at the arbitrary splice point. Additionally, the bit occupancy in the virtual buffer determined by the amount of compressed picture data in the elementary inserted stream is made to coincide with that in the second elementary stream in the virtual buffer at the arbitrary splice point.
As a feature of the present invention, the bit occupancy in the first virtual buffer for an elementary inserted stream is made to coincide with the bit occupancy in the first virtual buffer for the first elementary stream at the arbitrary splice point. The bit occupancy in the last virtual buffer attributable to the elementary inserted stream is similarly made to coincide with the bit occupancy in the virtual buffer for the second elementary stream at the arbitrary splice point. The elementary stream is inserted between the first and second elementary streams to generate a continuous compressed picture data stream.
As another feature of the present invention, when the bit occupancy in the virtual buffer by the elementary inserted stream exceeds the size of the virtual buffer, a pseudo data larger than the excess data for which the bit occupancy exceeds the virtual buffer size, is inserted between the first and second elementary streams so that the actual buffer utilized during decoding will not overflow.
According to the present invention, the compressed picture data and an elementary inserted stream, in which the pseudo data is inserted, are inserted between the first and second elementary streams to generate a continuous compressed picture data stream that will not overflow or underflow a buffer utilized during decoding.
The invention accordingly comprises the several steps and the relation of one or more of such steps with respect to each of the others, and the apparatus embodying features of construction, combinations of elements and arrangement of parts which are adapted to effect such steps, all as exemplified in the following detailed disclosure, and the scope of the invention will be indicated in the claims.